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(57) Abstract 

Computer software (150) and an associated digital camera system (100) for capturing digital wide angle such as hemispherical images, 
for processing two or more wide angle or hemispherical images to seam the wide angle or hemispherical images together to form a spherical 
image, for linking sound data files (105), location data (110), time and date data (100b) and horizontal reference data to the captured 
image data, for selecting for display (175) or for printing (165) undistorted portions of the spherical images (170), for posting the spherical 
images to a web site (160) or transmitting the spherical images via the Internet (155), for time and date stamping (100b) digital wide angle 
images as they are captured along with horizontal reference data, location data, for example, from the global positioning system (110), and 
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APPARATUS, MEDIA AND METHOD FOR CAPTURING 
AND PROCESSING SPHERICAL IMAGES 
BACKGROUND OF THE INVENTION 

1. Technical Field 

The present invention relates generally to the field of digital image capture and 
processing and, more particularly, to apparatus, media and a method for capturing 
digital wide angle such as hemispherical images, for processing two or more wide 
angle or hemispherical images to seam the wide angle or hemispherical images together 
to form a spherical image, for linking sound data files, location data, time and date 
data and horizontal reference data to the captured image data, for selecting for display 
or for printing undistorted portions of the spherical images, and for posting the 
spherical images to a web site or transmitting the spherical images via the Internet. 

2. Description of the Related Arts 

In US Patent No. 5,185,667, a hemispherical image processing system is 
described. The system comprises a camera, a fish eye lens and a digital processing 
system. The digital processing system operates on a data file containing data 
representing a distorted fish eye lens image and performs a transformation on the data 
to permit display of a perspective corrected image portion of the distorted fish eye 
image in real time responsive to pan, tilt, rotate and zoom inputs provided by a user. 

In US Patent No. 5,384,588 and 5,877,801, a wide angle image transmission 
system is described. For example, a camera equipped with a wide angle lens may 
capture images having a great field of view. Multiple remote users may input 
commands in real time in such a manner that each remote user may view different 
perspective corrected portions of the same hemispherical image. When a camera, for 
example, is capturing a single hemispherical image of a baseball game, one remote user 
may zoom in on the pitcher and view a perspective corrected image of the pitcher while 
another viewer may zoom in on the batter and view a perspective corrected image of 
the batter. Meanwhile, the camera has not moved as it captures the whole field of view 
of the baseball game. 

In US Patent 5,764,276, there is described a system of obtaining spherical 
views of an image as earlier described by US Patent 5,130,794. One may obtain a 
spherical image by capturing two hemispherical images. Two fisheye objective lenses 
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such as the NIKON Iras having a 220 degree field of view cooperate to capture a 
spherical image. The '276 patent further describes a camera positioning and rotator 
device for properly positioning a camera on a tripod and the digital processing of the 
spherical image to provide perspective corrected views in response to a user using a 
cursor for controlling pan, tilt, rotate and zoom. 

Cameras are now digital and capture images in the form of digital data. Most 
cameras are not equipped with wide angle or fisheye lenses. Fish eye lenses, in 
particular, are very large and very expensive (such as the NIKON 220 degree lens 
described above). The Internet is now a convenient network for the inexpensive and 
popular transmission of digital data. The transmission of e-mail has become 
commonplace and its use is encouraged by camera manufacturers for the transmission 
of digital images so, for example, family members may view digital pictures taken 
minutes earlier at a wedding. 

On the other hand, camera manufacturers do not currently supply inexpensive 
fish eye lenses. Complex image processing software is required for dewarping 
perspective corrected views in real time and for seaming wide angle images together. 

Consequently, there remains in the art a need to decrease the costs and simplify 
Ae processes associated with permitting a photographer to take advantage of the wide 
angle image technology briefly described above. 
SUMMARY OF THE INVENTION 

According to the principles of the present invention, a photographer may obtain 
a digital camera equipped with a normal field of view lens and a converter/adapter for 
such a lens that permits the photographer to capture a greater than 180 degree field of 
view with a single picture. Moreover, the photographer may utilize a rotating adapter 
for a tripod to which the digital camera, normal lens and fish eye lens 
converter/adapter are attached. The rotating tripod adapter mounts the camera in such 
a way as to support the focal or nodal point of the lens over the rotating axis of the 
adapter. The camera may turn and temporarily lock into positions exactly 180 degrees 
apart so as to capture opposing views as taught by pending US Application Serial No. 
08/767,376 filed December 16, 1996 and entitled "Method and Mechanism for 
Automatic Opposing Alignment of Photographic Image Capture. " The digital camera 
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of the present invention is further provided with a digital clock for date and time 
stamping each wide angle image as it is captured, horizontal reference data sensors for 
sensing and recording pan, tilt and rotation reference data for each wide angle image 
as it is captured and location data sensing circuitry such as global positioning system 
data sensors for obtaining and recording location data such as latitude and longitude 
data for each wide angle image as it is captured for forwarding to image processing 
apparatus according to the present invention. 

A wedding picture may be taken by asking the wedding party to gather around 
a digital camera equipped and mounted to the tripod as described above. One digital 
image is captured, the camera is then immediately rotated 180 degrees and a second 
digital image is captured. Pending US Application Serial No., filed concurrently 
herewith, (attorney docket no. 1096.79941), describes an imaging system in which 
both images are captured simultaneously and several wedding images captured in video 
sequence for viewing as home video virtual reality experiences. 

The hemispherical images captured in sequence are preferably time and date 
stamped and stored with horizontal reference data and camera location data temporarily 
in the digital camera for downloading to a digital image processor such as a personal 
computer on which is installed software according to the present invention. The 
hemispheres axe then downloaded into memory of the personal computer on which has 
been preinstalled software from a media for performing camera system calibration, 
automatic seaming, previewing the seamed spherical images, associating background 
or introductory audio files with seamed images, posting images to a web site with 
downloadable viewer software so they may be viewed by a remote viewer, bundling 
the images with a viewer, for example, in an e-mail so a family member recipient of 
the e-mail may view them, and customizing the spherical image by permitting the user 
to edit text or otherwise customize a tripod cap portion of the image. Each step of the 
image process may be provided in a convenient screen comprising a mode map area, 
a work area and a navigation area for navigating through the mode. Also, the present 
invention involves a method and system for doing business involving the purchase of 
keys for storing of seamed images. 

These and other features of the present invention will become better understood 
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through the drawings and detailed description of the invention which follows. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a system overview showing the individual elements of apparatus, 

media and method for capturing and processing spherical images including a wide angle 
S digital image capture system, digital audio data files, global positioning system data, 

a data processing system such as a personal computer, and the various output 

capabilities including Internet e-mail, image posting to a web site, printing 

conventional still images via a printer (black and white or color) in a selected aspect 

ratio, displaying the images and storing the images in a photo album memory. 
10 Figure la provides a block schematic diagram of a digital camera imaging 

system according to the present invention. 

Figure 2 show individual elements of the wide angle digital image capture 

system including a fisheye or wide angle camera lens adapter 220-1 (Figure 2e), a 

rotator 210 (Figure 2b), a tripod 200 (Figure 2a), a digital camera 215, 220 or 225 
15 equipped with the fisheye lens converter/adapter 220-1 or a digital camera already 

equipped with a fisheye lens. 

Figures 3a, 3b and 3c show in sequence the steps of taking opposing 

hemispherical images from a table or desk top wherein Figure 3a shows an assembled 

Olympus D340-L digital camera arrangement, the camera with a fisheye 
20 converter/adapter mounted to a rotator, and the rotator coupled to the tripod 200; 

Figure 3b shows a camera rotated on rotator 210 to capture a first hemisphere; and 

Figure 3c shows the camera rotated 180 degrees to capture a second hemisphere. 

Figure 4 is a screen print of a typical computer display screen for selecting a 

camera during software installation. 
25 Figure 5 provides an explanation of a calibration screen for calibrating a camera 

imaging system including a tripod and rotator. 

Figure 6 provides one example of a preview screen for previewing a spherical 

image formed by seaming two hemispheres. 

Figure 7 is a first screen of several screens for an image processing mode for 
30 processing digital image data, the first screen directed to collecting digital wide angle 

image data for one, two or more images. 
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Figure 8 shows the result from Figure 7 of selecting one image and using that 
one image to form a 360 degree round image. 

Figure 9 shows the more typical example where a 360 degree round image is 
to be formed from two opposing hemispheres. 

Figure 10 provides an overview of image downloading screens for downloading 
image data from digital camera memory, with and without TWAIN software. 

Figure 11 is a preview IPIX image screen for selecting desired IPIX image 
format for preview and adding keys as may be necessary for storing IPIX images in the 
selected format. 

Figure 12 is an add key screen accessed from the preview IPIX screen of Figure 

11. 

Figure 13 is a sound screen for selecting background and introduction sound 
data files for association with the IPIX image currently being processed. 

Figure 14 is a biographical information screen that may be completed by the 
user or photographer to, for example, name the IPIX image, associate key words with 
the image and identify the photographer. 

Figure 15 is a tripod cap screen, the tripod cap being a wasted portion of an 
IPIX image which may be used to convey information or to link the IPIX image being 
processed to a URL. 

Figure 16 is a close-up view of a default tripod cap supplied with the image as 
viewed from the preview screen of Figure 6. 

Figure 17 is a horizontal reference data screen whereby the user may manually 
enter pan tilt and rotate data associated with a particular image as referenced to true 
plumb or select horizontal (normal) or vertical camera orientation. 

Figure 18 is an image processing selection screen whereby a user may select 
one or more forms of image seam processing and at least one associated parameter. 

Figure 19 is the camera configuration screen for selecting a camera 
configuration. 

Figure 20 is another copy of the preview screen of Figure 6 useful for 
explaining the process of establishing zoom parameters and setting an initial IPIX 
image point of view. 
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Figure 21 shows the screen for storing an IPDC image after it has been 
processed and identifying file names for each desired format. 

Figure 22 is an example of a combined mode screen where all the screens of 
Figures 7-19 are combined into one screen which may be referred to herein as an 
5 expert mode screen. 

Figure 23 is an example of a panorama print screen accessible from an IPDC 
image viewer. 

DETAILED DESCRIPTION OF THE DRAWINGS 

Apparatus for capturing and processing spherical images according to the 

10 present invention is shown in Figure 1. A wide angle digital camera system 100 
according to the present invention comprises the several elements shown in Figure 2 
(digital camera, wide angle lens converter/adapter, tripod and camera mount and 
rotator) and may further including digital audio files 105 previously captured or 
recorded and stored simultaneous with the capture of a digital image and global 

IS positioning input data 110, preferably stored at the time of image capture. 

It is also preferable to record data representing pan, tilt and rotate coordinates 
referred to herein of the digital camera and camera rotator of Figure 2 with respect to 
true plumb as horizontal reference data 100a at the time of image capture. Typically, 
the digital camera may be assumed to be pointed in a plane horizontal with the plane 

20 of the ground or the floor. However, such is not always the case when capturing an 
image. According to the present invention, such reference level measurements may be 
made manually, however, with improvements in digital cameras, such measurements 
may be automatically registered as digital data along with the image captured in 
memory. For example, a small gyroscope and sensor may be used to register the 

25 coordinates in relation to camera direction and location at the time of image capture 
and in each opposing direction of camera aim as would be known from the surveying, 
aviation and maritime arts. 

Similarly, time of day and date clock data 100b may be preferably recorded 
simultaneously with image capture. Time of day and date may replace the use of 

30 image number to record image sequences and, of course, provides greater detail. 
Today's digital camera systems may not incorporate a digital clock for time and date 
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stamping of images as they are taken. Nevertheless, such time and date data is 
preferably recorded in conjunction with or auxiliary to image capture. So while today 
it may be necessary to manually record the time and date of image capture and continue 
to use an image sequence number as provided by conventional camera film, digital 
cameras of the future may simultaneously record time and date as each picture is taken 
with the digital image data files as time and date data. 

Digital audio files 105 may be prerecorded, post-recorded or computer 
composed or generated audio for providing accompanying music or explanatory 
language or both (voice over music) for a captured digital image. The digital files may 
also comprise simultaneously recorded audio and a digital microphone and digital 
recording functionality may be built in to digital camera system 100. Preferably, a 
digital or electret microphone may record audio data and be mounted and directed in 
the same direction as a camera lens to capture the sounds coming from the direction of 
image capture as would be known from the movie film and television programming 
studio arts. For a spherical image according to the present invention, it is preferable 
to provide for a stereo sound file, recording both the forward and opposing camera 
directions as left and right stereo audio channels. 

Likewise, global positioning system data 110 may be captured simultaneously 
with the image or by p rerecording or post-recording the location data as is known from 
the surveying art. The object is to record the precise latitude and longitude global 
coordinates of each image as it is captured. Having such data, one can easily associate 
front and back hemispheres with one another for the same image set (especially when 
considered with time and date data). Hie path of image taking from one picture to the 
next can be permanently recorded and used, for example, to reconstruct a picture tour 
taken by a photographer when considered with the date and time of day stamps. 

Other data may be automatically recorded in memory as well (not shown) 
including names of human subjects, brief description of the scene, temperature, 
humidity, wind velocity, altitude and other environmental factors. These auxiliary 
digital data files associated with each image captured would only be limited in type by 
the provision of appropriate sensing and/or measuring equipment and the access to 
digital memory at the time of image capture. One or more or all of these capabilities 
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may be built into wide angle digital camera system 100. 

As has been introduced in US Patents 5,764,276; 5,877,801 and US Patent 
Application Serial No. 08/767,376 filed December 16, 1996, the object of the image 
capture system 100 of the present invention is to capture two or more wide angle 
images, for example and preferably, two hemispherical images taken with a fish eye 
1ms camera and seam the two hemispheres together to form a spherical environment. 
A hemisphere as described and used herein is an approximately 360 degrees round by 
180 degree image (sometimes considered as 180 degrees by 180 degrees in latitude and 
longitude) that will contain distortion caused by the fish eye lens or lens system and, 
when viewed as substantially large flat image portions, will be perspectively incorrect. 
Two hemispheres when digitally seamed together using software media 150 according 
to the present invention form a spherical environment. The user of software 150 may 
view perspectively correct smaller portions and zoom in on those portions from any 
direction as if the user were in the environment, causing a virtual reality experience. 

When a fish eye lens is used such as a NIKON fisheye lens, the field of view 
that is captured is 220 degrees, which is 40 degrees more than a hemisphere. There 
is considerable overlap in opposing images that are captured. The overage permits 
inaccuracy in someone's inadvertently not capturing perfectly opposing views. 
However, these 220 degree field of view lenses are very large and very expensive. 
According to the present invention, a far less expensive and smaller 185 degree lens 
converter/adapter may be used to adapt a normal flat image lens purchased with a 
digital camera to capture a 185 degree field of view image. However, better 
registration of opposing hemispheres is required. With 185 degrees field of view, there 
is only 5 degrees beyond 180 degrees of overlap with the opposing hemisphere. 
Consequently, the camera system of the present invention uses a tripod and rotator 
mount for the camera that permits almost perfectly opposing hemispheres to be 
captured. 

Digital processing system 125 need not be a large computer. For example, the 
digital processor 125 may comprise an IBM/PC-compatible computer equipped with 
a Microsoft WINDOWS 95 or 98 or WINDOWS NT 4.0 or later operating system. 
Preferably, the system 125 comprises a quad-speed or faster CD-ROM drive, although 
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other media may be used such as Iomega ZIP discs or conventional floppy discs. An 
Apple Computer manufactured processing system 125 should have a MACINTOSH 
Operating System 7.5.5 or later operating system with QUICKTIME 3.0 software or 
later installed. The user should assure that there exists at least 100 megabits of free 
hard disk space for operation. An Intel Pentium 133 MHz or 603e PowerPC 180 MHz 
or faster processor is recommended so the spherical images, hereafter referred to as 
IPIX (TM) images where IPIX is a trademark or Interactive Pictures Corporation of 
Oak Ridge, Tennessee, may be seamed together and stored as quickly as possible. 
Also, a minimum of 32 megabits of random access memory is recommended. 

The personal computer should have a display 125-1 for adequately displaying 
high resolution IPIX spherical images. The display 125-1 or 175 should be capable of 
displaying 800 x 600 pixels or higher of resolution with at least 256 different colors. 
Processing system 125 should also comprise a keyboard input 125-3, a mouse and pad 
125-4 for inputting data selections. Of course, voice recognition may be supported in 
future releases of the processing software stored on media 150. Processor housing 125- 
2 comprises hard disc memory, cd-rom and other media drives, random access memory 
and processors as is well known in the art. 

Image processing software is typically produced as software media 150 and sold 
for loading on digital signal processing system 125. Once the software according to 
the present invention is properly installed, a user may load the digital memory of 
processing system 125 with digital image data from digital camera system 100, digital 
audio files 105 and global positioning data and all other data described above as desired 
and utilize the software to seam each two hemisphere set of digital images together to 
form IPIX images. 

Once the images are seamed and the user is pleased with the result, the user 
may output the IPIX images in various ways as will be further described herein. The 
IPIX images may be transmitted with a viewer as Internet e-mail 155 to addressed 
terminals where the e-mails may be opened and enjoyed by the recipients. The IPIX 
images may be posted on the user's web site 160. IPIX image portions that have been 
personally selected for content by the user may be transmitted to a preferably color 
printer for copying. (One family wedding IPIX image may yield a virtually unlimited 



WO 00/62542 PCT/US99/07667 

collection of conventional flat still images of the wedding.) 

The viewer may store the IPEX image in a database 180, for example, as a data 
folder collection of files called wedding. Also, the viewer may immediately or at any 
time view the IPIX image portions of his choice by panning, rotating, tilting and 
5 zooming to the image of choice and displaying the image on display 125-1 or display 
175 . Other outputs of the image data may come to mind of one of ordinary skill in the 
art. 

Figure la provides a block schematic diagram of a digital camera imaging 
system according to the present invention. A fisheye lens adapter 220-1 converts a 

10 typical digital camera lens 220-5 to a wide angle lens. In many instances the adapter 
220-1 may slidably fit and be friction attached to lens 220-5. Digital camera body 225 
may contain the elements 100a, 250, 255, 260, 265, 270, 275, 280 and 285 comprising 
a block schematic diagram of electronic digital circuitry. Some of the elements may 
be externally mounted but operate in synchronization with the image capture operation 

15 as will be described further below. Controller 250 typically comprises a small 
microprocessor or application specific integrated circuit. Programmable read only 
memory, array memory or other program memory houses the programs that operate 
the camera. Clock 260 is typically a digital clock that clocks the microprocessor 
operation. According to the present invention and possibly coupled with appropriate 

20 computer programs of program memory 270, clock 260 provides for time and date 
stamping of digital images as they are captured. Shutter 265 operates to permit an 
image to be captured via lens system 220-1, 220-5 and transmitted to digital image 
memory 275 which also stores other concurrently stored data such as the time and date 
stamp data. In accordance with the present invention, horizontal reference data 100a 

25 is sensed by well known sensors in the art and also captured for each image that 
represents the camera's pan, tilt and rotation angles with respect to predetermined 
tripod reference points 180 degrees apart and true vertical or plumb where pan is look 
around, tilt is up and down and rotate is turning the camera sideways. Location data 
such as latitude and longitude for each image is captured and stored for each image by 

30 location data sensor 255 which may be a global positioning system. Interfaces 280 and 
285 are provided for interfacing the captured image and other data with a computer 

10 
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system 125. 

In operation, when the user operates the shutter or preferably sets a timer so 
that the user may get in the picture and the timer expires, several operations occur 
simultaneously: time and date stamp, location data stamp, and horizontal reference data 
stamping occur simultaneously with image capture. In an embodiment not shown, 
digital camera 100 may be equipped with a means of entering biographical information 
as well as will be further described herein. Moreover, the digital camera may be 
equipped with at least one microphone for recording a simultaneous audio clip with the 
digital image of predetermined duration. 

Referring to Figure 2, there are shown the collection of elements comprising 
a typical wide angle digital camera system 100 according to the present invention. 
Figure 2a shows a tripod 200 according to the present invention. Ideally, a digital 
camera 225 should be mounted on a monopod, a one legged camera mount. Using a 
monopod reduces the size of a typically undesirable "footprint" made when capturing 
two images, a tripod cap image portion. An IPDC spherical image captures the sky, 
the full surroundings but, because of the tripod, leaves a small circular area at the 
bottom of an image that comprises the tripod 200 and not the floor or ground which 
might be a more desirable part of the image than the tripod 200. (The monopod or 
tripod appearance at the bottom of an image, the small wasted portion of the IPIX 
image, will be hereinafter be referred to as the tripod cap.) 

Tripod 200 comprises conventional elements including three tripod legs, 201a, 
201b and 201c. These are typically extendable. It is desirable in shooting a normal 
landscape and to simulate a true virtual reality experience to capture an image at the 
same level a person would capture the image at eye level. So the legs preferably 
extend to approximately five feet. Extendable height adjustment 202a in its non- 
extended position may preferably comprise a suction cup 202a or other device, for 
securely fastening the tripod on table tops or to level surfaces when the surface is not 
level Fastener 204 such as a screw may be used to hold the adjustment 202 in place. 
Tripod cap 203 is adapted to receive a rotator assembly 210 according to Figure 2b. 

Referring to Fig. 2b, there is shown a rotator 210 according to the present 
invention. The base 210a of rotator 210 is fixed to tripod cap 203 by a fastener that 
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may be a screw mount, bayonet mount or other fastener known in the art. In a 
preferred embodiment, the rotator 210 attaches to tripod 200 through screw heads of 
rotator base 210a. The rotator 210 is held by the rotator base 210a, turned clockwise 
and may spin slightly as it is screwed securely to the tripod 200. Once the rotator 210 

5 is attached to the tripod 200, the rotator arm 210b should be turned until it clicks softly 
into a detente position. Base 210a and rotating arm 210b are permitted to rotate one 
with respect to the other to two predetermined detente positions which are, in a 
preferred mode, exactly 180 degrees apart. A camera, for example, camera 215, 220 
or 225, are mounted at mounting point 210c which is a predetermined distance d from 

10 the axis of rotation of rotator 210. Referring briefly to Figures 2f and 3a to 3c, it can 
be seen how the camera lens is positioned directly over the rotator arm 210b and 
mounted so as to point along its length. Moreover, the focal node of the lens is 
mounted directly over the axis of rotation. Since cameras may vary as to the distance 
between the camera body and the lens, mount 210c may be slidably adjustable in the 

15 lengthwise direction of the rotator arm 210b to permit the lens to be mounted over the 
axis of rotation. The camera body is attached to the rotator arm 210b using, for 
example, a thumb screw. The camera should be snug along a back stop. It is 
important, in order to obtain quality IPIX images, to be sure that the tripod and rotator 
are firmly assembled together with the digital camera and are fixed and do not move 

20 with relation to the surface on which they rest, as one hemisphere is captured and then 
the second hemisphere is captured. For wide angle lens cameras, there would be more 
detente positions and more image seaming as necessary to capture a full 360 by 360 
image if desired. For example, three detente positions may be 120 degrees apart and 
four detente positions may be 90 degrees apart. Stereoscopic IPIX image capture is 

25 described by Patent Cooperation Treaty application PCT/US98/21304 entitled "Method 
for Creation and Interactive Viewing of Totally Immersive Stereoscopic Images. " 

Typical digital cameras 215, 220 and 225 are shown in Figures 2c, 2d and 2f 
respectively. Camera 215 is an Olympus D-340L with a fisheye lens converter 215-1, 
215-2 shown mounted over the conventional Olympus lens beneath. Mounting bracket 

30 215-2 must be used to mount the fisheye lens converter 215-1 to the camera body. 
With the Kodak DC200 camera 220 shown in Figure 2d, the fisheye lens converter 
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220-1 slidably mounts over the existing lens and is held in place by friction. Figure 
2f shows a NIKON Coolpix 900s with the NIKON fisheye lens converter mounted over 
the NIKON lens. The camera is shown already mounted to a rotator 210. Other 
cameras may be used to advantage as well so long as a fisheye or wide angle lens 

5 converter may be adaptably fit to the camera or the fisheye or wide angle lenses are 
available for the camera. The digital cameras ideally capture digital images with as 
high a resolution as possible. Resolution is typically quantified in terms of a pixel 
array such as 600 by 800. The larger the number of pixels in each of horizontal and 
vertical dimensions, the better the resolution and the better quality of IPIX image that 

10 one can zoom into. In accordance with US Application serial no. 08/835,210, entitled 
"Method and Apparatus for Providing a High Resolution Image to Produce a Realistic 
Immersion Experience," as one zooms into a portion of an image such as a painting on 
a wall, a high resolution flat still image may be replaced for the perspective corrected 
portion of the image zoomed into. 

IS Figure 2e shows a 185 degree field of view fisheye lens adapter 220-1 and 

carton 220-2 manufactured by FIT Corporation of Tokyo, Japan. The lens converter 
converts the optics of a standard 1ms to be able to capture a full 185 degree field of 
view, permitting overlapping hemispherical images to be captured for seaming together 
as an IPIX spherical image. 

20 Now the practice of capturing hemispherical images will be discussed with 

reference to Figures 3a, 3b and 3c. An IPIX spherical image is an interactive, 
spherical image with a complete field of view, from earth to sky, floor to ceiling, 
horizon to horizon. An IPIX image is created from two opposing photographs captured 
by a fisheye lens. The present invention remaps the images, seaming them into an 

25 immersive whole. IPIX images can be linked together, linked to traditional 
photographs and rendering, enhanced with audio or other file types and incorporated 
into other applications as will be further described herein. IPIX images can be applied 
in a variety of ways including but not limited to virtual world creation, computer-based 
training, real estate sales and marketing, museum tours and travel logs. As will be 

30 described herein, IPIX images may be posted on a World Wide Web site for viewing 
by industry-standard web browsers or with an embedded IPIX viewer, e-mailed for 
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remote viewing by a local IPIX viewer, imported into Adobe Photoshop software 
applications for enhancement or touch-up, embedded into a Microsoft PowerPoint or 
MacroMedia Director and Authorware presentations or viewed from an Microsoft 
Visual Basic, Visual C or Visual C++ application program. 

Once two hemispheres are captured to form an IPIX spherical image, software 
media ISO is run on a processor 12S to seam the hemispheres together, color match the 
images to adjust for different exposures in the opposing views, blend the seam to create 
a single image out of the two hemispheres, save the IPIX image on one or more sizes 
and resolutions, configure the IPIX image with sound files and brand the IPIX with 
personal information or environmental information through the design of a tripod cap 
(the bottom of the image). 

An IPIX image has two hemispheres, front and back. It is preferable to shoot 
consecutive image pairs without moving the tripod. Composing an IPIX image 
involves two major decisions; the image's point of view and where a seam will fall. 
The point of view should be decided by deciding where to place the camera and 
tripod/rotator. IPIX images place the viewer in the center of a picture and can make 
objects appear farther away than they really are. 

The seam location can be chosen to avoid human subjects or the work of art 
center of an image. Desirably the seam should fall in an area of not much interest. 
For example, the camera is taking the picture of the family at dinner with the camera 
mounted in the center of the table. Place the camera so it takes half the people in one 
image (with no one at the seam) and half the people in the other image. The scene 
should be free of unwanted objects. It is preferable to use a high quality mode if the 
camera provides for it and to disable the flash. Most cameras have a timer to give the 
user a few seconds to either be completely in or out of the picture as desired and avoids 
camera movement (since no one is pushing on a shutter button). 

After taking one hemisphere, the camera is rotated to the opposite position. 
Referring to Figure 3a, the camera 215 is shown mounted on the tripod/rotator 200. 
The suction cup 202a may be used to hold the tripod to the table top . Figure 3b shows 
the camera tripod 200 and rotator 210 rotated to a first detente position facing right. 
Then one rotates the rotator in direction 310 (or the opposite direction) to obtain the 
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opposite position shown in Figure 3c facing left. 

IPIX images can be taken without a tripod or rotator but are susceptible to 
blurring from camera movement, or there may be problems with aligning the two 
hemispheres if there is insufficient overlap. One way is to mark a spot on the ground 
and an arrow so the camera user can try to emulate a tripod/rotator as closely as 
possible, holding the camera level in both directions. 

Now installation of software media 150 will be briefly described followed by 
calibration of the software with the camera of the user's choice. Thereafter, the 
features of the method and media will be further described in connection with a 
discussion of the operation of the software on a pair of hemispheres. 

Installation of the image processing software of the present invention, 
hereinafter referred to as IPIX Wizard software, involves well known software media 
150 installation steps. One inserts the IPIX Wizard CD-ROM into the CD-ROM drive 
of data processing system 125 and a Setup program should start automatically. One 
may click on "next" or "agree" through a succession of Welcome and Readme screens. 
One finally arrives at a destination screen wherein the user must identify a location in 
memory for the software to reside. 

Once a destination is selected, the screens according to Figure 4 appear on 
display 125-1. Here the user chooses the type of camera the user will be using with 
the software. Preferably, a choice is required before the user may "continue." After 
a choice of camera is made, the IPIX Wizard files and IPIX Viewer files may be 
copied to the hard drive 125-2. If an Olympus camera was selected, TWAIN drivers 
will be installed by a secondary installation program. TWAIN drivers and their 
importance to receiving digital image files will be explained further herein. Further 
Readme and registration screens may follow as is well known in the art. 

Camera calibration requires the input of data files for two hemispheres captured 
using the camera selected during installation of the software media 150 for the IPIX 
Wizard software. Figure 5 shows a calibration screen for leading a user through the 
calibration process. The calibration process calculates the seam location from two 
images taken with the camera system including rotator and tripod the user wishes to 
calibrate. IPIX Wizard software can contain default configurations for each of the 
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camera types supported by the software. However, when the user calibrates their 
camera with IPEX Wizard software, a new calibrated camera configuration file is 
generated. The camera calibration file appears in Figure 19 of the IPK Wizard mode 
and Figure 22 of the Expert mode as will be described further herein. 

Camera calibration can be accessed in one of two ways: from the Preferences 
screen or when IPK Wizard software prompts the user to calibrate. As already 
indicated, it is necessary to download hemispheres from the camera to the computer. 
The camera manual will have instructions for attaching most cameras to a computer for 
downloading. On an Open Hemispheres screen one can click to locate the images (it 
does not matter which image is selected first) and these are entered into the calibration 
screen of Figure 5 as images 501 and 502. Box 503 shows the user's camera type and 
configuration. There arc two choices for calibration: quick and thorough per box 504. 
It is recommended that the user try a quick calibration and, if the result is acceptable, 
it is likely that die thorough could not have done any better. The quick calibration may 
be as much as twenty-five times faster than the thorough calibration. A time of 
calibration screen may appear showing an estimate of the amount of time remaining in 
the calibration. 

The program first makes an initial guess at the center of each hemisphere to 
locate the border. Then the program places one image border in a linear or circular 
buffer and attempts to match the image border of the other hemisphere to it. When 
there is an optimum match, then the seaming is complete. For example, a first pixel 
at the coiter of the initial center image guess is checked and then several new neighbors 
checked in either the x or y coordinate system direction to see if an optimum has been 
readied yet. In one embodiment a circle find algorithm may improve the guess made 
of the center of a circle and other embodiments, two linear buffers may be used or 
circular buffers with slops for unimportant portions of die image (such as the tripod cap 
portions). 

Once calibration is complete, one can preview the image formed by the seaming 
process. If the results are unacceptable, one may use the thorough calibration 
selection. A typical preview screen is shown in Figure 6. The image seam is shown 
at the arrow in the previewed perspectively correct view intentionally selected at the 
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border between two hemispheres 602 and 603. 

Now the operation of IPIX Wizard software will be further explained with 
reference to the several screens of Figures 7-22. Figures 7-21 represent the IPIX 
Wizard mode and Figure 22 represents all the IPIX Wizard mode screens in one and 
is referred to herein as the Expert mode. Once a user becomes knowledgeable of the 
processes described below for the IPIX Wizard mode, then, the user may use the 
Expert mode screen of Figure 22. The first screen of the IPIX Wizard mode is an open 
hemispheres screen according to Figures 7-10 which involve opening one or more 
hemispheres to be seamed. There exist three portions of this screen. An IPIX Wizard 
map shows the particular screen of the IPIX Wizard mode that is currently displayed 
or active. By clicking on one or more of the choices, one may access screens of the 
IPIX Wizard mode out of sequence as desired. The Open Hemispheres screen is 
presently highlighted in Figure 7. There is also a work area 702 and a navigation area 
703. The work area is now used to input two hemispheres for operation. Portion 704 
permits the choice of a single hemisphere or two hemispheres. A single hemisphere 
can be seamed to its reverse image for a visual effect if desired as taught by US 
Application Serial No. 08/516,629 filed August 18, 1995. This option is shown in 
Figure 8 as "Mirror this hemisphere" choice 801. On the other hand, for a true virtual 
reality experience, two hemispheres should be selected and loaded per Figure 9. In 
Figure 9, the user can click on double arrow button 905 to swap front and back images 
if appropriate. The file names are shown in the boxes below the respective front and 
back hemisphere images. a No thumbnail" in Figure 7 represents that no image has 
been selected yet. One can browse to locate the image via browse buttons 903 and 904 
or type the file name into the respective boxes 901 or 902. The "Acquire'' button 906 
is used to acquire images from your camera, provided your camera has a TWAIN 
driver. TWAIN stands for Technology (or Toolkit) without an interesting name. A 
TWAIN driver is apiece of software that interacts with a scanner or digital camera to 
allow computer 125 to communicate with the device. If the camera does not have a 
TWAIN driver, then direct camera access cannot be obtained; the user may download 
the images before starting the IPIX Wizard mode. Figure 10 shows what will happen 
if the user clicks on "acquire" and there is or is not a TWAIN driver installed. If the 
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TWAIN exists for the camera, for example, the Kodak DC 200, then images may be 
directly selected. 

Referring to Figure 11, Preview IPK is highlighted in the map area 701. 
There are four choices provided for image preview - high resolution, low resolution, 
JAVA and e-mail. (JAVA is a registered trademark of Sun Microsystems, Inc. of 
Mountain View, California). The high resolution choice saves an IPIX image in a 
1024 x 1024 pixel file. The low resolution choice saves the image as 512 x 512 pixels. 
The JAVA option saves a 1400 x 700 pixel image. 

The JAVA format may be posted to a web site with a runtime downloadable 
JAVA viewer applet. The difference between JAVA applet viewer software and 
browser software such as a Netscape or Microsoft Explorer (ActiveX) plug-in viewer 
is that the JAVA applet is typically smaller and simpler. Also, JAVA-capable browsers 
such as Internet Explorer and Netscape Navigator download the applet with the web 
page upon which it resides, much the same way as the browser downloads the other 
elements of a web page. The JAVA applet simplifies the user experience by 
eliminating the need for the user to download and install a plug-in with their browser. 

The e-mail option saves an IPIX image as an executable file bundled with a 
viewer. The result does not need to be e-mailed to someone but can be used to create 
local presentations. If transmitted as an e-mail, the IPIX image can be viewed on a PC 
running Windows 95/98 and NT or by an Apple MACINTOSH OS computer. Other 
formats in other resolutions are also possible and may come to mind of one of ordinary 
skill in the art and may be preferred in some applications. IPIX image files have a 
.IPX extension by convention and the e-mail IPIX image file has a .EXE extension. 
Add keys button 1101 is also shown. 

Figure 12 relates to the on-line purchase of keys for IPIX images. If the user 
needs keys (needed to save an IPIX image), the user uses this screen to purchase keys. 
A key is not required to run the IPK Wizard image build process, but a key must be 
available at the end of the process to save the seamed IPK image to memory. 

Selection of the Add Keys button 1101 on the Preview IPIX screen (Figure 11) 
will install new keys or verify current IPK image key count. Figure 12 shows the Add 
Keys screen. A ate ID code 1201 may be unique to each computer installed with IPK 
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Wizard software. The code 1201 is required to purchase additional keys. For 
example, the site code 1201 may comprise twelve characters, ten unique to the site and 
two for uniquely identifying the software. IPIX keys field 1202 (shown as 0 keys 
remaining) shows the number of remaining keys from those already purchased and 
remaining. IPIX software typically may be purchased with a predetermined number 
of keys. When new keys are purchased, a 30 digit code 1203 appears in the screen that 
may be used to activate the new toys that have been downloaded. The user raters the 
code and clicks on ok to activate the keys. A user may also redeem key certificates at 
an IPIX web site store by entering the certificate number at the web site. 

Figure 13 relates to the addition or association of sound files to an IPIX image. 
In the Wizard map area 701 , Sound is highlighted. In the depicted screen, area 1301 
permits the selection of an audio background to the IPIX image. This may, for 
example, be a prestored MIDI (musical instrument digital interface) file or other file 
that has been computer composed or generated (for example, MIDI) or recorded 
contemporaneous with the image capture or before or after image capture. It may be 
concert music or other audio data background to the playing of the image. Area 1302 
permits the selection of a sound file to be played as an introduction to the IPIX image. 
A user might want to prerecord or voice synthesize a description of the image before 
it is shown to a viewer, for example, a VOX or WAV file. WAV file support is built 
into Windows 95 and VOX files are compressed audio data files (optimized for speech) 
and downloaded easily. 

Figure 14 relates to the customization of an IPIX image by adding bio or other 
personal information. Bio Information is highlighted in the map area 701. Also, Bio 
1400 is one of four tabs that may be selected for customizing an IPIX image. There 
are three ways shown for adding personal bio information. The user may name the 
IPIX image in area 1401, identify key words in area 1402 and name the photographer 
in area 1402. The latter area may be utilized to generate along with time and date data 
a copyright notice protecting the IPIX image. Other ways may come to mind of a user. 
For example, one may manually input time and date data, location data, environmental 
data (temperature, humidity and so on) as described above. 

Figure 15 is entitled tripod cap in map area 701. Also, tripod cap is selectable 
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as tab 1501. Tripod cap relates to that portion of an IPIX spherical image taken up by 
the tripod, typically at the bottom of an IPIX image. One may create their own tripod 
cap. For example, one may draw a square and use a program such as MS Paint 
software to design a custom cap to be saved as a .bmp or TIFF file. Using the browse 
button, the user selects the customized cap they have created. Also, a user may place 
a link on their tripod cap that, when actuated, takes control of the web browsers to the 
specified hotspot at URL link area 1502. In alternative embodiments, the link may be 
to any file type, for example, to link audio to the image or "hotspot* from one IPDC 
image to another IPIX image in an image sequence. 

Figure 16 shows a close-up of the tripod cap provided automatically in an IPIX 
image. The default tripod cap may be, for example, 350 x 350 pixels. 

Figure 17 relates to Ref pan/tilt/rotate in map area 701 and Ref PTR in the 
customizing tabs 1701. The choices include Horizontal, Vertical and User defined. 
The typical tripod arrangement will be horizontal. That is the camera is set up 
vertically to shoot horizontally. If the camera points at the sky for a first hemisphere 
and the ground for the second hemisphere, then, vertical orientation should be shown. 
If the user is not on even ground, then the user may input the coordinates for pan, tilt 
and rotate. It is also suggested according to the present invention to embed a sensor 
in the body of a camera for automatically recording pan, tilt and rotate measurements 
in relation to each hemisphere captured. The sensor may be taken from the well known 
aviation or maritime arts. 

Figure 18 relates to image options in map area 701 or image processing 1801 
in the customizing tabs section. There exist four choices: blend seam and a selectable 
range of pixel counts, color balance hemispheres, image smoothing and adjust image 
alignment. When an IPIX image is created, the two hemispheres' seams are blended 
together. The user may increase or decrease the number of pixels being blended 
together. Increasing or decreasing may improve the final image quality. A default 
value may be 15 pixels with reference to the overlap of five degrees if the lens as a 185 
degree field of view. Seam blending finds pixels which are assumed to match in the 
overlap area between the two hemispheres. It blends the two hemispheres by taking 
a weighted average of the matching pixels. The weighting is applied as a function of 
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the distance from the seam or edge of the hemisphere in towards the selected pixel 
value, for example, 15 pixels. 

Image smoothing is really interpolated scaling. For the various available 
formats, the source hemispheres are scaled either in a fast pixel picking interpolate 
mode or a slower pseudo-bilinear interpolate mode known in the art. Also, a color 
balancing routine may be run to smooth the relative color exposures between front and 
back images. The color balancing routine can eliminate large color shifts between 
hemispheres. Hie color balancing algorithm finds pixels that are assumed to match in 
the overlap area between the two hemispheres. It averages the difference in each of 
the three color components (red, great and blue) between the matching pixels. It then 
scales the colors in each hemisphere in such a manner as to eliminate the perceived 
difference between the matching pixels. The result of color balancing is compensation 
for mismatched exposures between hemispheres. 

Omitting these steps will result in faster seaming time but decrease image 
quality. Image smoothing can provide a smoother IPEX image but may double the 
creation time. If the user wants to calibrate the hemispheres before building the IPIX 
image from them, the calibrate routine may be called by selecting "Adjust image 
alignment. * 

Figure 19 shows the camera configuration screen. The camera configuration 
is identified in box 1901. 

Figure 20 shows a typical image preview screen that is displayed of an IPIX 
image after the steps above have been performed of seaming, blending, balancing and 
the like. One may pan, tilt, rotate and zoom within the image but to save the image 
to a file requires a key. Within this screen, the user may set minimum and maximum 
zoom (magnification levels) values from predetermined parameters, change the initial 
viewpoint (the view that is seen when the IPIX is first opened), save the IPIX image 
and reset the image to a default. To change a viewpoint, the user pans or tilts around 
the image to find a viewpoint and selects the Set View button 2001. At any time the 
user may return to the initial view by clicking a go to initial view" 2002 or right 
clicking with the mouse. The Set Zoom Min 2003 and Set Zoom Max 2004 buttons 
are used to change the minimum and maximum magnification values from 
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predetermined zoom values to new minimum and maximum limits. Go to Zoom 
buttons 2005 and 2006 allow return to the predetermined zoom limits. Finally, reset 
to default button 2007 allows the user to reset everything to default values. 

Figure 21 allows one to save an MX image in one of many formats. The first 
major area is the save area 2100 used to identify and store the IPIX image file. One 
can check any one or more of several formats: Initial view thumbnail 2101 , hemisphere 
thumbnail 2102, high resolution IPIX 2103, low resolution IPIX 2104, JAVA IPIX 
2105, e-mail IPIX 2106 and e-mail IPIX 2107 with spin (pan and tilt). One or more 
of these boxes may be selected for the price of one key. Thumbnails do not cost a key. 
Cancel allows the user not to save the IPIX image that was created. 

Once many IPIX images have been saved, an album (folder) can be created in 
memory of an event or a vacation or whatever. G.P.S. location data and time and date 
stamps may be used to sequentially tell a story as appropriate accompanied by music 
and introductory audio. Each IPIX image may have an infinite number of different 
views. For example, a wedding picture of A and B may be taken comprising a single 
IPIX image at the church. A print at printer 165 may select the married couple or the 
whole wedding party including bride's maids and ushers. Other views may be selected 
of some of the guests in different groupings. A high quality color printer 165 is 
preferred and produces as many flat still images 170 as the viewer wishes from the 
same IPIX. 

Theprint process will be further described with reference to Figure 23. A user 
accesses a view and pans, tilts, rotates and zooms to a desired perspective corrected 
view. A parameters pull-down menu may be used to select an aspect ratio and a print 
size such as 8" by 10" or other size (such as is known in the art). By aspect ratio is 
intended herein the ratio of image width to image height of a flatimagel Forexample, 
atypicalaspectratiomaybeltol,4by3,or5by3. A panoramic aspect ratio may 
have a higher aspect ratio than 5 by 3. By size is intended the print size of the image 
such as eight inches by ten inches. 

In accordance with the present invention, a warning screen may be used to warn 
a user that the dots per inch resolution of the printer may not provide a presentable 
image if too large a size is selected. Forexample, if the printer resolution is only 100 
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dots per inch and an eight inch by ten inch image size is selected, the image will not 
be clear and the warning message may be indicated. 

Warning messages may also be provided if too large of the surface area of a 
sphere has been selected for printing. In mapping a sphere to a cylinder and all the 
vertical dimension is selected, the upper and lower portions of an image will be 
stretched. As aspect ratio changes and image size increases to capture more of the 
vertical or horizontal, and more of the sphere is intentionally captured for printing, the 
resulting view will not be perspective corrected in the sense that a human viewer would 
not believe that the image is perspectively correct. For example, if one attempts to 
print a spherical view of the earth, the resulting view of the sphere mapped to a 
cylinder would show the North and South Poles as exaggerated in size (a typical 
equirectangular image mapping). With reference to the Earth, one would be able to 
select approximately 45 degrees above and below the equator (90 degrees total) without 
stretching the resulting image too greatly at the top and bottom. The resulting printed 
image would appear perceptively correct to a human. Consequently, an error message 
may be used to indicate that a selected image for printing will not be perspectively 
correct to a viewer if printed when the selected vertical range is above 90 degrees. 

In one embodiment, the user pulls down a menu including Print from clicking 
on the File button 2301 of Figure 23. The Print menu may include, for example, a 
panorama choice, a fit the page choice (the largest possible image) and a small size 
choice (such as a standard photographic size like three inches by five inches which also 
represents a five by three aspect ratio). 

Figure 23 provides an example of a screen which one would see if one selects 
panorama viewing from an IPIX viewer software screen. When a user arrives at this 
screen they may pan, tilt, rotate and zoom through the IPIX spherical image to select 
a central most image section for panoramic printing. Thus, the screen of Figure 23 
may depict in the background the selected center of the panoramic image. A 360 
degree image is generated in the horizontal dimension for the printed panorama but a 
range of values for field of view 2302 from 60 degrees to 130 degrees is provided for 
selection in the vertical dimension. As one changes the selected field of view value, 
the screen may contain in a window (not shown) a preview of the selected panorama 
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image (not shown). 

Many cameras today permit the capture of a panoramic or wide angle image 
having, for example, a variable aspect ratio in the horizontal dimension. While a 360 
degree image is generated in the present, depicted embodiment, the user in another 
embodiment may select the horizontal dimension as well. The horizontal dimension 
may be selected from viewing and cropping a preview, printable image or by selecting 
a horizontal dimension parameter by selecting degrees (for example, 240 degrees), by 
aspect ratio or by image print size, for example, three inches by eight inches, or 
combinations of these approaches. 

Also, IPIX images may be operated upon, for example, to extract a selected 
perspectively correct or other portion and save the file so that it may be edited or 
otherwise processed for display or otherwise (add computer generated graphics, used 
as a background set for live or cartoon characters and the like). The saved file may be 
retrieved by a second software package for computer graphics generation and addition 
to the image and the like or such applications may be included in the IPIX Wizard 
software. 

Finally, a sequence of IPIX images may be linked and stored to form a 
conventional video file, for example, by storing an expanding file over time as a 
viewer pans, tilts, rotates and zooms their way through a single IPIX image file or one 
that has been linked by hotspots to other IPIX image files. A tour may comprise a 
collection of IPIX images and audio files in sequence, may be generated from a single 
IPIX image or may be collected over time as a real time 360 degree video experience. 

Thus there has been described apparatus, media and a method for capturing 
digital wide angle such as hemispherical images, for processing two or more wide 
angle or hemispherical images to seam the wide angle or hemispherical images together 
to form a spherical image, for linking sound data files, location data, time and date 
data and horizontal reference data to the captured image data, and for selecting for 
display or for printing undistorted portions of the spherical images. Posting the 
spherical images to a web site or transmitting the spherical images via the Internet are 
further described by an appendix which assumes the reader is familiar with HTML 
(Hypertext Markup Language). All patents and patent applications referenced herein 
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should be deemed to be incorporated by reference as to their entire contents. The 
following United States patent applications are related by subject matter to certain 
aspects of the present invention: US Patent Application serial no.'s 08/373,434 filed 
January 17, 1995; 08/516,629 filed August 18, 1995; 08/835,210 filed April 7, 1997; 

5 08/863,584 filed May 27, 1997; and 09/093,337 filed June 9, 1998. Any and all 
United States patents or patent applications referenced herein should be deemed to be 
incorporated by reference as to their entire contents. Further variations and 
modifications of the present invention may come to mind of one of ordinary skill in the 
art through an appreciation of the above description of the invention which should not 

10 be deemed to be limited by the scope of the claims which follow. 
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APPENDIX 

SETTING UP A WEB SITE 

To post IPIX images to the World Wide Web, a user must create a Web page 
with special HTML code, and, when using an IPIX Plug-in software module, configure 
a server MIME type. This section assumes the user has a basic understanding of the 
Hypertext Markup Language (HTML). 

MIME TYPES 

When IPIX image files are used on Web pages, the approximate MIME type 
must be configured on the Web Server. On the local computer, the MIME type is 
configured when the IPIX Plug-in software module is installed. The MIME type must 
also be set on the hosting server (a one time configuration) IPIX images will not be 
displayed properly. 
What are MIME Types? 

IPIX image files present a new file type to systems that have not run them 
before. Whenever a new file type is introduced, the operating system must be 
instructed how to handle it and what application is used process it. 

MEME types are used to help define the new file type and application software 
used to process it. This applies to both Web browsers and Web servers. 
IPIX Image MIME Types On Web Servers 

When a Web server is not properly configured for the IPIX IMAGE MEME 
type, users experience difficulties in downloading and viewing IPIX image files. When 
the .IPX extension is detected but is not on the Web server MIME type list, a default 
application type and subtype will be employed. The browser will attempt to assign an 
application to open the IPIX image file. Because no default application can open an 
IPIX image file, the user will not be able to view the IPIX image on their browser. 

Configuring a MIME type for a Web server is usually not difficult. 
• The server maintains a file indicating the types of documents that correspond 
to various file extensions. The name of this file differs with each Web server. 
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• When the server gets a request for a URL leading to a file which has one of 
these extensions, it first sends a single line to the browser which states the 
major and minor MIME types. 

The Webmaster or Internet Sendee Provider will be able to set up the MIME 
types on the server hosting the user's Web pages. 

The MIME types that should be added to the Web server are: 



IMIME type/subtype 


Extension 1 


| application/x-ipix 


file extension is .IPX | 


|| application/x-ipscript 


file extension is .IPS I 



To establish the MIME types: 

• Install the IPIX Plug-in software module on the user's computer to establish the 
MIME types on the local machine. 

• Make sure the computers hosting the user's Web pages have the proper MIME 
types set. Again, ask the Web master or Internet Service Provider (ISP) to add 
the appropriate MIME types to all of their servers. 

If an IPIX image displays in Internet Explorer, but not in Netscape (via 
Netscape plug-in viewer software), the MIME types are probably not set up 
properly. 

THE IPIX PLUG-IN AND VIEWERS 

Interactive Pictures Corporation has developed automated and semi-automated 
viewer downloading and installation. The degree of automation depends upon browser 
type and operating system. There should exist both a visible link to the IPIX Plug-in 
software module download, and using an embedded object tag with the IPIX image on 
the Web page. 

These tags are not necessary if the user is using JAVA IPIX images. JAVA 
IPIX images contain a built-in IPIX Viewer, and do not require the IPIX Plug- 
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in. (JAVA is a registered trademark of Sun Microsystems of Mountain View, 
CA) 

Linking to the IPIX Plug-in Download 

To aid site visitors without an IPIX Plug-in, provide a link to the Interactive 
Pictures Corporation's Plug-in download area: http://www.ipix.com/cgi- 
bin/download.cgi. One option is to use an IPIX (Interactive Pictures Corporation) logo 
as a graphic linked to the download page. 

• With clickable text: 

< A HREF= w http://www.ipix.com/cgi-bin/download.cgi w > 
Clickhere. </A> 

• With a clickable graphic: 

< A HREF= ,, http://www.ipix.com/cgi-bin/download.cgi ,> > 
<EMG SRC = "images/ipixlogo. gif BORDER =0 
ALIGN = "MIDDLE" > </A> 

Embed Object Tag for Plug-ins Page 

In addition to providing a link to the download page, embedding the IPIX image 
with an object tag can help users obtain the correct Plug-in. This code sample will 
open a link to the Plug-in page for people who do not have the IPIX Plug-in installed. 

• The first three lines of code are ActiveX commands (where ActiveX is 
the name of the plug-in viewer software) for Microsoft Internet 
Explorer that identify the type of Object (IpixXl) the Class and version 
number, and the location of the IPIX Plug-in software module to be 
installed (Codebase). 

• PARAM NAME provides the ActiveX component with the IPIX image 
filename. 

• The Last four lines associate the file type with IPIX file, and the 
location of the IP IX Plug-in page if the application associate with x-ipix 
is not found. 

< OBJECT n)= n IpixXr WIDTH =300 HEIGHT=400 

28 



00/62542 



PCT/US99/07667 

CLASSID="CLSD:1 1260943-421B-1 1DO-8EAC-O000C07D88CF 



CODEBASE=°ht^://ww.ipix.com/viewers/ipixx.caMversion=2,2,0,r> 
<!-ForMSIE3,4+-> 

<PARAMNAME='TPXFILENAME" VALUE = "y ourimage. ipx " > 
< !- For Netscape 3,4 - > 

< EMBED SRC ="yourimage.ipix" HEIGHT =300 WIDTH =400 
PALETTE = "foreground" BORDER =0 
type = "application/x-ipix* 

pluginsPage ="http://www.ipix.com/cgi-bin/download.cgi' , > 

What happens when the HTX Plug-in Software Module is downloaded? 

It depends upon what browser and operating system are on the computer. If the 
user has Netscape 4.x or Internet Explorer 3.x/4.x on a Windows-based PC, then 
download and installation is almost invisible. The Plug-in is installed with the browser 
still running. If die user is using a Macintosh browser, or an older version of Netscape 
or Internet Explorer, the user will have to download the installation program to their 
hard drive, close the browser, and then run the installation. 

FILE LOCATIONS AND NAMING CONVENTIONS 

When me user is placing files on the server, the user must meet the following 
requirements: 



1 Controlled IPIX 
Image Size 


To specify the size of the IPIX image 
within the browser 


Full Screen 


To display full screen IPIX images 


JAVAScript Window 


To display a full screen, controlled 
size IPIX image in a new window. 1 
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• The IPIX images must be stored in the same directory with any 
associated scripts, sound files, and multimedia components used by an 
IPIX image. 

• The IPIX images must be in the same directory in the HTML or in a 
subdirectory. 

• It is strongly advised to keep all file names and directory names in 
lowercase characters. 

• When the user displays an IPIX image in a frame set, you must use the 
frame name in all UPPERCASE letters. This is a requirement for the 
frames to function properly. 

• The naming convention used for the JAVA IPIX Viewer is 
a IpixViewer.class, w where the I and V are capitalized. 

Absolute path names are supported, but relative paths are 
recommended. Performance is usually improved when relative paths 
are employed. 

There are three ways to place IPIX files into HTML code for viewing: 

Controlled IPIX Image Size 

When the user wants to control the IPIX image size within the browser window, 
employ the EMBED SRC tag. 

To set IPIX image size, include the HEIGHT and the WIDTH dimensions 
within the EMBED SRC tag. 

It is also essential to include the PALETTE attribute with the value foreground: 
[PALETTE= w foreground w ]. 

Full Screen Image 

The simplest way to post an IPIX image for full screen viewing is to use the 
HREF Anchor tag. When this method is used, the IPIX will fill the browser window. 

The following HTML code samples can provide some guidance when one is 
building Web pages. 
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To view an IPIX image: 

• With a clickable graphic another Web page: 

<A HREF w ht^://wwapix.com/ipixx/14344.ipx" > Click here < IMG 
SRC = "images/yourimage.gir WIDTH =100 
HEIGHT =86 BORDER =0 < >/A> 

• With a clickable graphic to a relative address: 

< A HREF "ipix/sample.ipx" > Click here <1MG 

SRC = "images/yourimage.gif" WIDTH =100 HEIGHT =86 
BORDER=0<>/A> 

• Without a clickable graphic: 

< A HREF ="ht$://ww.ipfr.com/ipix/sample.ipx'' Click here < /A > 
<IMG SRC = "images/yourimage.gir WIDTH =100 HEIGHT =86 
BORDER =0> 

• Using FRAMES. The HTML code using a FRAME which contains an 
IPIX link: 

< FRAME SRC = "http://www.ipix.com/ipix/yourimage.ipx" 
name=TPJX"> 

JAVAScript Window 

It is possible to view IPIX images in separate windows. The size of this 
window is controlled. This is accomplished using JAVAScript. 

Some browsers, including Internet Explorer 3.x and AOL 4.0, may have 
problems viewing pages using JAVAScript. 

The following code may be used for this: 

< !-THJS SCRIPT OPENS IPIX WINDOWS- > 
</HEAD> 

< SCRIPT LANGUAGE= "JavaScript" > 
function OpenlpixWindow (ipix_path) 

{ 
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window.open(*ipix/'+ipixjMth, 'IPDCWin', / 

4 height=300,width=400 , ); 

} 

</SCRIPT> 
</HEAD> 
<BODY> 
<A 

HREF="javascript:OpenIpixWindow ('yourifnage.ipx');'' > 

Click to View IPIX Image</A> 

</BODY> 

WORKING WITH JAVA IPIX IMAGES 

JAVA IPIX images are designed for viewing with the IPIX JAVA Viewer. It 
is unnecessary for a customer to download the IPIX Plug-in software module to view 
a JAVA IPIX image. They are, however, generated in a lower resolution to reduce file 
size and may not be acceptable in all cases. 

The IPIX JAVA Viewer files consist of the JAVA class files and an archive of 
the JAVA applet (IpixViewer.jar). JAR files work with Microsoft Internet Explorer 
and Netscape Navigator 4.0 and higher. If someone viewing your page has a 3.x 
version of a browser, the JAR file will not be recognized, and instead, all of the class 
files will be loaded. This has the same effect as the JAR file, but it takes a little bit 
longer to display the page. 

For the latest IPIX JAVA Viewer updates, visit the Interactive Picturers' FTP 
site at f^://ftp.omniview.com/public/viewers/java. 

JAVA Requirements on the Web Server 

There are four elements that must be placed on the Web server to properly post 
a JAVA IPIX image: 

• The HTML file that links to the JAVA IPIX image; 

• The IPIX JAVA Viewer itself, named IpixViewer.jar; 
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• The JAVA class files; and 

• The JAVA IPIX image intended for viewing. 

The JAVA IPIX images and the JAVA Viewer may be in the same 
folder/directory and on the same server. 

JAVA Web Page Requirements 

To use JAVA IPIX images with the IPIX JAVA Viewer on a Web page follow 
these guidelines: 

• The Spin value can be on or off. The JAVA applet allows a page 
designer to set the JAVA image to an auto-spin mode by simply 
changing the value of the Spin parameter to on. 

• The image size can be changed, and the Width value may be a multiple 
of four (more or less). 

Creating a JAVA Web Page 

Capitalization is very important. If the user does not have the proper 
capitalization, the page will not work correctly. This information is discussed in "File 
Locations and Naming Conventions" earlier in this chapter. 

When the user creates their page and post it to an Internet site, the user will 
need to upload the class files. These files are included with your IPIX Wizard. If the 
user installed their IPIX software to the default location, then the JAVA Viewer files 
are located in the IPIX software folder, and on the IPIX CD. 
Resizing The Page 

Resizing a Web page containing a JAVA IPIX image may cause problems with 
image display. When the user resizes the image, the browser automatically refreshes 
the Web page to reformat it to the new display size. The IPIX image may display as 
a black screen. To correct this, the user should hold down the Shift button on then- 
key board and click the Refresh button on the browser. The IPIX image should display 
properly after the page reloads. 

POSTING IPIX IMAGES TO THE WEB 
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These steps will walk the user through posting IPIX images to a Web page. 
Everything from MIME types to JAVA/Non-JAVA IPIX images are covered. 

STEP 1: ESTABLISHING MIME TYPE SETTINGS 

Make sure the server system or Internet Service Provider has the necessary 
MIME type settings. For IPIX images to work with the user's server system, these 
MIME type settings are necessary: 

application/x-ipscript file extension=ips 

application/x-ipix file extension=ipx 

For more information on MIME types, see "MIME Types", on page 28 of this 
Appendix. 

STEP 2: GATHERING ALL NECESSARY FILES 

Make sure all of the IPIX image files— IPIX images (.IPX), IPScripts (.IPS), 
and audio files (.MID, .VOX, .WAV) are together in one directory. 

STEP 3: EMBEDDING THE IPIX IMAGE 

Embed the IPIX image onto the user's Web page. The code used to embed an 
IPIX image on a Web page varies depending on how one prefers the IPIX picture to 
appear. If the user would like the IPIX image to open to 640 x 480, the user should 
use the following code: 

For JAVA IPIX Images, go to Step 5 (Using JAVA IPIX images). 

<a href = w yourimage.ipix n > something clickable like text or a 

button </a> 

If the user would like to control the opening size of the IPIX image, use the 
following code, varying the width and height values: 

< OBJECT ID= tt IpixXr WIDTH =370 HEIGHT =290 
CLASSID ="CLSID: 1 1260943-421B-1 1D0-8EAC-0000C07D88CF* 
CODEBASE="http://www.ipix.^ 
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<!-For MSIE3,4+ -> 

<PARAMNAME=TPXFJLENAME" VALUE ="yourimage.ipx" > 
<!— For Netscape 3,4 — > 

< embed src = "yourimage. ipx" border =0 width =320 
height =240 palette = "FOREGROUND " < > /embed > 
</OBJECT> 

STEP 4: CONTROLLING THE IMAGE SIZE 
To open an IPIX image full screen: 

<a href="yourimage.ipx"> something clickable like text or a 

button < /a > 

To control the size of the IPIX image: 

< OBJECT ID ="IpixXr WIDTH =370 HEIGHT =290 
CLASSID ="CLSID: 1 1260943-42 IB- 1 1D0-8EAC-0000C07D88CF" 

CODF^ASE="ht^://www.ipix.com/viewers/ipixx.cab#version=2,2,0,l"> 
<!-ForMSIE3,4+ -> 

<PARAM NAME = "IPXFILENAME" VALUE = "yourimage. ipx" > 
< !- For Netscape 3,4 -> 

< embed src="yourimage.ipx" border =0 width— 320 
height=240 palette = "FOREGROUND" < > /embed > 
</OBJECT> 

STEP 5: USING JAVA IPIX IMAGES 

Embed the JAVA IPIX image onto the user Web page using the following 
JAVA applet. With this code, the user must have the .jar and the .class files in the 
same folder as the HTML page that uses this code. 

< applet code = "IpixViewer . class " 
archive= "Ipix Viewer .jar" 

align = "baseline" 
width ="320" 
height="240" 
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name = "Ipix Viewer" > 

<param name = "Spin" value="6n" > 

<param name = "URL w value= ,, you^image.ipx ,, > 

</applet> 

If the user has multiple JAVA IPIX images on their Web site, the user can add 
a codebase line to the user's applet. Using the codebase line allows the user to keep 
the .jar and .class files in one place and to access this one location from any page on 
the user's Web site. 

< applet code^IpixViewer.class" 

archive="IpixViewer.jar* 

codebase="http://www,yourdomainxom/java w 

align = "baseline* 

width= w 32(r 

height^^* 

name = "Ipix Viewer" > 

<param name == "Spin" value^on" > 

<param name = "URL" value =="yourimage.ipx w > 

< /Applet > 
Explanation of Tenns: 

The following items are explanations of terms used in the above sample of 
HTML code. 

• APPLET tag is required to call the JAVA applet. 

• ARCHIVE indicates the name and location of the JAVA applet. 
The IpixViewer.jar is a compressed form of the JAVA applet. 

• CODEBASE indicates the absolute or relative location of the 
JAR and class files. This tag is only required if the IPIX JAVA 
Viewer and the IPIX images are in different locations. 

• WIDTH=320 This indicates the width of the applet in pixels. 
320 is the suggested size. The Width value may be divisible by 
four (more or less). 
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• HEIGHT =240 This indicates the height of the applet in pixels. 
240 is the suggested height. 

• PARAM NAME = "URL" This parameter indicates the name 
and the location of the JAVA IPDC image. Here the parameter 
is VALUE^yourimage.ipx". This indicates that the JAVA 
IPDC image is named yourimage.ipx and is located in the same 
directory as the HTML file that is calling it. 

• PARAM NAME = "Spin" This parameter turns on and off the 
automatic spin feature of the JAVA IPIX image. The VALUE 
may be set to ON or OFF. When the Spin parameter is not 
specified, it defaults to off. 

For more information on the applet tag, visit Sun's JAVA Web site at: 
http://java.sun.c»m/docs/books/^ 
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WHAT WE CLAIM IS: 

1. Apparatus for processing a plurality of digital wide angle images comprises a 
digital camera system and an image processor for seaming together said plurality of 
wide angle images to form a 360 degree round image and for associating audio files to 
said 360 degree round image, said plurality of digital wide angle images having an 
audio channel associated with each said wide angle image such that, with two such 
digital wide angle images, a first digital wide angle image has an associated left stereo 
channel and a second digital wide angle image has an associated right stereo channel. 

2. Apparatus as recited in claim 1 wherein said audio files include an introductory 
audio prior to viewing said 360 degree round image. 

3. Apparatus as recited in claim 1 wherein said audio files represent background 
audio during viewing of said 360 degree round image. 

4. Apparatus as recited in claim 1 wherein said digital wide angle images comprise 
hemispherical images. 

5. Apparatus as recited in claim 1 wherein said digital camera system comprises 
a time clock for digitally time stamping said digital wide angle images upon image 
capture. 

6. Apparatus as recited in claim 4 wherein said digital camera system comprises 
a standard lens and a fisheye lens converter. 

7. Apparatus as recited in claim 1 wherein said digital camera system comprises 
a location sensor for outputting location data for storage with said wide angle image 
as it is captured. 

8. Apparatus as recited in claim 1, said image processor for receiving data for 
customizing a wasted image portion according to user input. 



38 



WO 00/62542 



PCT/US99/07667 



9. Apparatus as recited in claim 1, said image processor for receiving data for 
identifying a hotspot for association with a selected image portion. 

10. Apparatus as recited according to claim 5 further comprising a user interface 
for inputting personal data, the image processor for constructing copyright indicia for 
said 360 degree round image responsive to said user interface and said clock. 

11. Apparatus as recited in claim 1 further comprising a printer, said image 
processor, responsive to user inputs in real time for selecting a portion of said 360 
degree round image, for outputting said selected image portion for flat still image 
printing. 

12. Apparatus as recited in claim 11 wherein said flat still image comprises a 
panoramic image portion of said 360 degree round image. 

13. Apparatus as recited in claim 1 for outputting said 360 degree round image for 
posting to a web site. 

14. Apparatus as recited in claim 1 for outputting said 360 degree round image as 
a self-executable file. 

15. Apparatus as recited in claim 14 wherein said self-executable file includes 
viewer software bundled together with the image. 

16. Apparatus for outputting a 360 degree round image for posting to a web site for 
association with a viewer software applet. 

17. Apparatus for processing a plurality of digital wide angle images comprising a 
digital camera system and an image processor for seaming together said plurality of 
wide angle images to form a 360 degree round image and for outputting said 360 
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degree round image for posting to a web site for association with a viewer software 
applet. 



18. Apparatus as iecited in claim 17, said image processor for further posting said 
image to a web site with associated audio files. 

19. A method for processing a plurality of digital wide angle images for use with 
a digital image processor, at least two of said digital wide angle image having an 
associated audio file, the method comprising the steps of seaming together said 
plurality of wide angle images to form a 360 degree round image and associating said 
audio files to said 360 degree round image as left and right stereo audio files in relation 
to a selected portion of said 360 degree round image for viewing. 

20. A method as recited in claim 19 further comprising the step of receiving data 
for customizing a wasted image portion according to user input. 

21. A method as recited in claim 19 further comprising the step of receiving data 
for identifying a hotspot for association with a selected image portion, the hotspot 
linking said 360 degree round image to a second 360 degree round image having 
different associated stereo audio files. 

22. A method as recited in claim 19 further comprising the steps of receiving 
personal and date stamp data and constructing copyright indicia for said 360 degree 
round image responsive to user command. 

23. A method for processing a plurality of digital wide angle images for use with 
a digital image processor comprising the steps, responsive to user inputs in real time, 
of selecting a perspectively correct portion of a 360 degree round image, the selected 
image portion having selected image aspect ratio and size, and outputting said selected 
image portion. 
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24. A method as recited in claim 23 further comprising the step of outputting said 
360 degree round image for posting to a web site. 

25. A method as recited in claim 23 further comprising the step of outputting said 
360 degree round image as a self-executable file for viewing. 

26. A digital camera comprising a controller, a digital image memory and a digital 
clock for outputting time of day and date data, said camera being equipped with a fish- 
eye lens for capturing first and second opposing hemispheres from the same camera 
location for forming a spherical image from said first and second opposing 
hemispheres, said controller, responsive to actuation of a shutter, for simultaneously 
recording the time of day and date for association with said captured digital spherical 
image in digital image memory. 

27. A digital camera comprising a controller, a digital image memory and a location 
data sensor for outputting camera location data, said camera being equipped with a 
fish-eye lens for capturing first and second opposing hemispheres from the same 
camera location for forming a spherical image from said first and second opposing 
hemispheres, said controller, responsive to actuation of a shutter, for simultaneously 
recording the location data for association with said captured digital spherical image 
in digital image memory. 

28. A digital camera comprising a controller, a digital image memory and a 
horizontal reference data sensor for outputting horizontal reference data, said 
controller, responsive to actuation of a shutter, for simultaneously recording the 
horizontal reference data for association with a captured digital image in said digital 
image memory. 

29. A digital camera system comprising a digital camera including a time of day 
and date clock, a lens and a wide angle lens adapter, said digital camera for mounting 
to a camera rotator having 180 degree opposing detente positions, said rotator for 
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mounting to a tripod having three legs and a fourth central leg, the fourth central leg 
having attachment means for attaching the tripod to other than horizontal surfaces. 

30. A digital medium for storing digital data representing a method for processing 
a plurality of digital wide angle images for use with a digital image processor, the 
method comprising the steps of seaming together said plurality of wide angle images 
to form a 360 degree round image and associating audio files to said 360 degree round 
image, said plurality of wide angle images having an audio channel associated with 
each said wide angle image such that with two such wide angle images, a first image 
has an associated left stereo channel and a second image has an associated wight stereo 
channel. 

31. A method for processing a plurality of digital wide angle images into a 360 
degree round image comprising the steps of storing in memory for association with 
each 360 degree round image an initial point of view and minimum and maximum 
magnification parameters which are different from default values for said initial point 
of view and maximum and minimum magnification parameters. 

32. A method for processing a plurality of digital wide angle images into a 360 
degree round image comprising the step of storing a selectable parameter value for a 
seam width between digital wide angle images and for selection data representing color 
balancing and image smoothing processes. 

33. A method for purchasing keys for permitting storage of 360 degree round 
images generated by an image processing system, the method comprising the steps of 

displaying a number of keys remaining to a user, 

displaying a unique site code for said image processing system and transmitting 
said unique site code to a key distribution center, and 

responsive to outputting an order for additional keys, displaying a unique key 
access code, 



WO 00/62542 PCT/US99/07667 



AMENDED CLAIMS 

[received by the International Bureau on 3 January 2000 (03.01.00); 
original claims 1,19 and 30 amended; new claim 34 added; 
remaining claims unchanged ( 6 pages)] 



1 1. Apparatus for processing a plurality of digital wide angle images comprises a 

2 digital camera system and an image processor for seaming together said plurality of 

3 wide angle images to form a single 360 degree round spherical image being capable of 

4 viewing as a sequence of perpsectively correct, flat image portions and for associating 

5 audio files to said single 360 degree round spherical image, said plurality of digital 

6 wide angle images having an audio channel associated with each said wide angle image 

7 such that, with two such digital wide angle images, a first digital wide angle image has 

8 an associated left stereo channel and a second digital wide angle image has an 

9 associated right stereo channel. 

1 2. Apparatus as recited in claim 1 wherein said audio files include an introductory 

2 audio prior to viewing said 360 degree round image. 

1 3. Apparatus as recited in claim 1 wherein said audio files represent background 

2 audio during viewing of said 360 degree round image. 

1 4. Apparatus as recited in claim 1 wherein said digital wide angle images comprise 

2 hemispherical images. 

1 5. Apparatus as recited in claim 1 wherein said digital camera system comprises 

2 a time clock for digitally time stamping said digital wide angle images upon image 

3 capture. 

1 6. Apparatus as recited in claim 4 wherein said digital camera system comprises 

2 a standard lens and a fisheye lens converter. 

1 7. Apparatus as recited in claim 1 wherein said digital camera system comprises 

2 a location sensor for outputting location data for storage with said wide angle image as 

3 it is captured. 
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1 8. Apparatus as recited in claim 1, said image processor for receiving data for 

2 customizing a wasted image portion according to user input. 

1 9. Apparatus as recited in claim 1, said image processor for receiving data for 

2 identifying a hotspot for association with a selected image portion. 

1 10. Apparatus as recited according to claim 5 further comprising a user interface 

2 for inputting personal data, the image processor for constructing copyright indicia for 

3 said 360 degree round image responsive to said user interface and said clock. 

1 11. Apparatus as recited in claim 1 further comprising a printer, said image 

2 processor, responsive to user inputs in real time for selecting a portion of said 360 

3 degree round image, for outputting said selected image portion for flat still image 

4 printing. 

1 12. Apparatus as recited in claim 11 wherein said flat still image comprises a 

2 panoramic image portion of said 360 degree round image. 

1 13. Apparatus as recited in claim 1 for outputting said 360 degree round image for 

2 posting to a web site. 

1 14. Apparatus as recited in claim 1 for outputting said 360 degree round image as 

2 a self-executable file. 
3 

4 15. Apparatus as recited in claim 14 wherein said self-executable file includes 

5 viewer software bundled together with the image. 

1 16. Apparatus for outputting a 360 degree round image for posting to a web site for 

2 association with a viewer software applet. 

1 17. Apparatus for processing a plurality of digital wide angle images comprising a 



44 

AMENDED SHEET (ARTICLE 19) 



□Tula i 



WO 00/62542 PCT/US99/07667 



2 digital camera system and an image processor for seaming together said plurality of 

3 wide angle images to form a 360 degree round image and for outputting said 360 

4 degree round image for posting to a web site for association with a viewer software 

5 applet. 

1 18. Apparatus as recited in claim 17, said image processor for further posting said 

2 image to a web site with associated audio files. 

1 19. A method for processing a plurality of digital wide angle images for use with 

2 a digital image processor, at least two of said digital wide angle image having an 

3 associated audio file, the method comprising the steps of seaming together said plurality 

4 of wide angle images to form a single 360 degree round spherical image being capable 

5 of viewing as a sequence of perspectively correct, flat image portions and associating 

6 said audio files to said single 360 degree round spherical image as left and right stereo 

7 audio files in relation to a selected portion of said single 360 degree round spherical 

8 image for viewing. 

1 20. A method as recited in claim 19 further comprising the step of receiving data 

2 for customizing a wasted image portion according to user input. 

1 21. A method as recited in claim 19 further comprising the step of receiving data 

2 for identifying a hotspot for association with a selected image portion, the hotspot 

3 linking said 360 degree round image to a second 360 degree round image having 

4 different associated stereo audio files. 

1 22. A method as recited in claim 19 further comprising the steps of receiving 

2 personal and date stamp data and constructing copyright indicia for said 360 degree 

3 round image responsive to user command. 

1 23. A method for processing a plurality of digital wide angle images for use with 

2 a digital image processor comprising the steps, responsive to user inputs in real time, 
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3 of selecting a perspectively correct portion of a 360 degree round image, the selected 

4 image portion having selected image aspect ratio and size, and outputting said selected 

5 image portion. 

1 24. A method as recited in claim 23 further comprising the step of outputting said 

2 360 degree round image for posting to a web site. 

1 25. A method as recited in claim 23 further comprising the step of outputting said 

2 360 degree round image as a self-executable file for viewing. 

1 26. A digital camera comprising a controller, a digital image memory and a digital 

2 clock for outputting time of day and date data, said camera being equipped with a fish- 

3 eye lens for capturing first and second opposing hemispheres from the same camera 

4 location for forming a spherical image from said first and second opposing 

5 hemispheres, said controller, responsive to actuation of a shutter, for simultaneously 

6 recording the time of day and date for association with said captured digital spherical 

7 image in digital image memory . 

1 27. A digital camera comprising a controller, a digital image memory and a location 

2 data sensor for outputting camera location data, said camera being equipped with a fish- 

3 eye lens for capturing first and second opposing hemispheres from the same camera 

4 location for forming a spherical image from said first and second opposing 

5 hemispheres, said controller, responsive to actuation of a shutter, for simultaneously 

6 recording the location data for association with said captured digital spherical image in 

7 digital image memory. 

1 28. A digital camera comprising a controller, a digital image memory and a 

2 horizontal reference data sensor for outputting horizontal reference data, said 

3 controller, responsive to actuation of a shutter, for simultaneously recording the 

4 horizontal reference data for association with a captured digital image in said digital 

5 image memory. 
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1 29. A digital camera system comprising a digital camera including a time of day and 

2 date clock, a lens and a wide angle lens adapter, said digital camera for mounting to a 

3 camera rotator having 180 degree opposing detente positions, said rotator for mounting 

4 to a tripod having three legs and a fourth central leg, the fourth central leg having 

5 attachment means for attaching the tripod to other than horizontal surfaces. 

1 30. A digital medium for storing digital data representing a method for processing 

2 a plurality of digital wide angle images for use with a digital image processor, the 

3 method comprising the steps of seaming together said plurality of wide angle images 

4 to form a single 360 degree round spherical image being capable of viewing as a 

5 sequence of perspectively correct, flat image portions and associating audio files to said 

6 single 360 degree round spherical image, said plurality of wide angle images having 

7 an audio channel associated with each said wide angle image such that with two such 

8 wide angle images, a first image has an associated left stereo channel and a second 

9 image has an associated right stereo channel. 

1 31. A method for processing a plurality of digital wide angle images into a 360 

2 degree round image comprising the steps of storing in memory for association with 

3 each 360 degree round image an initial point of view and minimum and maximum 

4 magnification parameters which are different from default values for said initial point 

5 of view and maximum and minimum magnification parameters. 

1 32. A method for processing a plurality of digital wide angle images into a 360 

2 degree round image comprising the step of storing a selectable parameter value for a 
3. seam width between digital wide angle images and for selection data representing color 
4 balancing and image smoothing processes. 

1 33. A method for purchasing keys for permitting storage of 360 degree round 

2 images generated by an image processing system, the method comprising the steps of 

3 displaying a number of keys remaining to a user, 
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4 displaying a unique site code for said image processing system and transmitting 

5 said unique site code to a key distribution center, and 

g responsive to outputting an order for additional keys, displaying a unique key 

7 access code. 

1 34. A method of decrementing one key from at least one purchased key comprising 

2 the steps of seaming together wide angle images to form a single 360 degree round 

3 spherical image and decrementing one key responsive to storing the single 360 degree 

4 round spherical image. 
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STATEMENT UNDER ARTICLE 19 

Responsive to the International Search Report in the above-titled International application 

issued on 04 November 1999 and prior to publication, please amend the application to replace 

S ^Mi- 
sheets ^6- >f with new sheets (pages) 26-32 comprising claims 1-34 and the recommended 

Abstract. Claims 1, 19 and 30 have been amended to recite "a single 360 degree round spherical 

image being capable of viewing as a sequence of perspectively correct, flat image portions." 

Also, a new independent claim 34 has been added to claim applicants' method of decrementing 

one from one purchased key. No changes were made to original sheets 32-43 (an Appendix); 

these sheets have been renumbered as sheets 33-44. 

REMARKS 

These remarks are made responsive to the Citations and Explanations section of the PCT 
International Search Report issued 04 November 1999 by Authorized Officer Tommy Chin (by 
James R. Matthew). The Officer indicates that Ritchey (US 5,130,794) is a document of 
particular relevance with respect to claims 1-4, 12, 19, 23, 26, 27 and 30. Applicants note a 
description of microphones 67 in Figure 6 and element 31 of Figure 8. Ritchey suggests that 
microphones 67 "face outward" (column 12, line 52) - on the "left and right side of the housing 
31 such that stereo sound is recorded" (column 12, lines 54-55). Ritchey fails to describe 

associating audio files with a single 360 degree round spherical image, said single image being 
capable of viewing as a sequence of perspectively correct, flat image portions of the spherical 
image. 

Independent claims other than claims 1, 19 and 30 have not been amended. These other 
independent claims including claims 16, 17, 23, 26-29 and 31-33 are directed to a number of 
concepts. Allegedly, for example, recording time of day and date (claim 26) and location (claim 
27) of a captured spherical image are shown by Ritchey. Ritchey discloses a conventional 
camcorder. The Examiner may be assuming that camcorders have time and date stamping. If 
so, the Examiner is encouraged to identify a teaching of same. Location (claim 27) is not 
something known to applicants and could not be located in Ritchey. Moreover, posting spherical 
images to a web site, preselecting aspect ratio or ordering keys off a web site, among other 
claimed features could not be located in the applied references. 

Applicant has added a new claim 34 directed to a method decrementing one key when a 
spherical image is stored. None of the cited references show such a method. 

Applicant respectfully requests that claims 1-34 be examined as this application either 
enters the national stage or applicants demand preliminary examination. 
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